Detecting Red Eye Filter and Apparatus Using Meta-Data

ABSTRACT

A method of filtering a red-eye phenomenon from an acquired digital image including a multiplicity of pixels indicative of color, the pixels forming various shapes of the image, includes analyzing meta-data information, determining one or more regions within the digital image suspected as including red eye artifact, and determining, based at least in part on the meta-data analysis, whether the regions are actual red eye artifact. The meta-data information may include information describing conditions under which the image was acquired, captured and/or digitized, acquisition device-specific information, and film information.

PRIORITY

This application is a Continuation of U.S. patent application Ser. No.11/690,834, filed Aug. Mar. 25, 2007; which is a Divisional of U.S.patent application Ser. No. 10/635,862, filed Aug. 5, 2003, now U.S.Pat. No. 7,630,006; which is a Continuation-in-Part application whichclaims the benefit of priority to U.S. patent application Ser. No.10/170,511, filed Jun. 12, 2002, now U.S. Pat. No. 7,042,505; which is aContinuation of U.S. patent application Ser. No. 08/947,603, filed Oct.9, 1997, now U.S. Pat. No. 6,407,777; which is hereby incorporated byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to digital photography usingflash, and specifically to filtering “Red Eye” artifacts from digitalimages shot by digital cameras or scanned by a digital scanner as partof an image acquisition process or an image printing process.

2. Description of the Related Art

i. Red Eye Phenomenon

“Red-eye” is a phenomenon in flash photography where a flash isreflected within a subject's eye and appears in a photograph as a reddot where the black pupil of the subject's eye would normally appear.The unnatural glowing red of an eye is due to internal reflections fromthe vascular membrane behind the retina, which is rich in blood vessels.This objectionable phenomenon is well understood to be caused in part bya small angle between the flash of the camera and the lens of thecamera. This angle has decreased with the miniaturization of cameraswith integral flash capabilities. Additional contributors include therelative closeness of the subject to the camera, iris color where lighteyes are more susceptible to this artifact and low ambient light levelswhich means the pupils are dilated.

The red-eye phenomenon can be somewhat minimized by causing the iris toreduce the opening of the pupil. This is typically done with a“pre-flash”, a flash or illumination of light shortly before a flashphotograph is taken or a strong additional light source. This causes theiris to close. Unfortunately, these techniques typically delay thephotographic exposure process by 0.5 second or more to allow for thepupil to contract. Such delay may cause the user to move, the subject toturn away, etc. Therefore, these techniques, although somewhat useful inremoving the red-eye artifact, can cause new unwanted results.

ii. Digital Cameras and Red Eye Artifacts

Digital cameras are becoming more popular and smaller in size. Digitalcameras have several advantages over film cameras, e.g. eliminating theneed for film as the image is digitally captured and stored in a memoryarray for display on a display screen on the camera itself This allowsphotographs to be viewed and enjoyed virtually instantaneously asopposed to waiting for film processing. Furthermore, the digitallycaptured image may be downloaded to another display device such as apersonal computer or color printer for further enhanced viewing. Digitalcameras include microprocessors for image processing and compression andcamera systems control. Nevertheless, without a pre-flash, both digitaland film cameras can capture the red-eye phenomenon as the flashreflects within a subject's eye. Thus, what is desired is a method ofeliminating red-eye phenomenon within a miniature digital camera havinga flash without the distraction of a pre-flash.

An advantage of digital capture devices is that the image contains moredata than the traditional film based image has. Such data is alsoreferred to as meta-data and is usually saved in the header of thedigital file. The meta-data may include information about the camera,the user, and the acquisition parameters.

iii. Digital Scanning and Red Eye Artifacts

In many cases images that originate from analog devices like film arebeing scanned to create a digital image. The scanning can be either forthe purpose of digitization of film based images into digital form, oras an intermediate step as part of the printing of film based images ona digital system. Red Eye phenomenon is a well known problem even forfilm cameras, and in particular point and shoot cameras where theproximity of the flash and the lens is accentuated. When an image isscanned from film, the scanner may have the option to adjust itsscanning parameters in order to accommodate for exposure and colorbalance. In addition, for negative film, the scanner software willreverse the colors as well as remove the orange, film base mask of thenegative.

The so-called meta data for film images is generally more limited thanfor digital cameras. However, most films include information about themanufacturer, the film type and even the batch number of the emulsion.Such information can be useful in evaluating the raw, uncorrected colorof eyes suffering from red eye artifacts.

iv. Red-Eye Detection and Correction Algorithms

Red-eye detection algorithms typically include detecting the pupil anddetecting the eye. Both of these operations may be performed in order todetermine if red-eye data is red-eye or if an eye has red-eye artifactin it. The success of a red eye detection algorithm is generallydependent on the success of a correct positive detection and a minimalfalse detection of the two. The detection is primarily done on imagedata information, also referred to as pixel-data. However, there isquite a lot of a-priori information when the image is captured and thenature of the artifact that can be utilized. Such information relies onboth anthropometric information as well as photographic data.

v. Anthropometry

Anthropometry is defined as the study of human body measurement for usein anthropological classification and comparison. Such data, albeitextremely statistical in nature, can provide good indication as towhether an object is an eye, based on analysis of other detected humanobjects in the image.

vi. Bayesian Statistics

A key feature of Bayesian methods is the notion of using an empiricallyderived probability distribution for a population parameter such asanthropometry. In other words, Bayesian probability takes account of thesystem's propensity to misidentify the eyes, which is referred to as‘false positives’. The Bayesian approach permits the use of objectivedata or subjective opinion in specifying an a priori distribution. Withthe Bayesian approach, different individuals or applications mightspecify different prior distributions, and also the system can improveor have a self-learning mode to change the subjective distribution. Inthis context, Bayes' theorem provides a mechanism for combining an apriori probability distribution for the states of nature with new sampleinformation, the combined data giving a revised probability distributionabout the states of nature, which can then be used as an a prioriprobability with a future new sample, and so on. The intent is that theearlier probabilities are then used to make ever better decisions. Thus,this is an iterative or learning process, and is a common basis forestablishing computer programs that learn from experience.

Mathematically,

While conditional probability is defined as:

${P\left( {AB} \right)} = \frac{P\left( {A\bigcap B} \right)}{P(B)}$

In Bayesian statistics:

${P\left( {AB} \right)} = \frac{{P\left( {BA} \right)}{P(B)}}{P(A)}$

Alternatively a verbal way of representing it is:

${Posterior} = \frac{{Likelihood} \times {Prioir}}{Normalizing\_ Factor}$

Or with a Likelihood function L( ), over a selection of events, which isalso referred to as the Law of Total Probability:

${P\left( {B_{i}A} \right)} = \frac{{L\left( {AB_{i}} \right)}{P(B)}}{\sum\limits_{{all} - j}^{\;}\; {{L\left( {AB_{i}} \right)}{P\left( B_{j} \right)}}}$

A Venn diagram is depicted in FIG. 8-b.

SUMMARY OF THE INVENTION

A method of filtering a red-eye phenomenon from an acquired digitalimage including a multiplicity of pixels indicative of color, the pixelsforming various shapes of the image, is provided. The method includesanalyzing meta-data information including information describingconditions under which the image was acquired and/or acquisitiondevice-specific information; determining one or more regions within saiddigital image suspected as including red eye artifact; and determining,based at least in part on said meta-data analysis, whether said regionsare actual red eye artifact.

The method may further include obtaining anthropometrical information ofhuman faces and the determining, based at least in part on saidmeta-data analysis, whether the regions are actual red eye artifact,being based further on the anthropometrical information.

The filtering may be executed within a portable image acquisitiondevice, having no photographic film. The filtering may be executed as apost-processing step on an external computation device.

The meta-data information describing the conditions under which theimage was acquired may include an indication of whether a flash was usedwhen the image was acquired and/or an aperture at the time of theacquisition. The acquisition device information may include sensor sizeand/or a spectral response of a sensor of the acquisition device. Theacquisition device information may further or alternatively include acolor transformation from raw sensor pixel values to saved image pixelvalues. A color of the pixels indicative of red eye color may becalculated based on a spectral response and a color transformation.

A lens may be used to capture the image. The meta-data information mayinclude a focal length of the lens and/or a focusing distance of thelens at time of acquisition.

The actual red eye artifact may be determined based on calculatedexpected size of the red eye artifact based on the meta-data informationincluding the acquisition device information. The calculated expectedsize of the red eye artifact may be defined as a range with a densityprobability function. The range may be determined by depth of fieldwhich is a function of said aperture setting. The method may furtherinclude obtaining anthropometrical information of human faces and thedetermining, based at least in part on the meta-data analysis, whetherthe regions are actual red eye artifact, may be based further on theanthropometrical information. The range may be determined by astatistical distribution of the anthropometrical information.

The determining whether the regions are actual red eye artifact may beperformed as a probability determination process based upon multiplecriteria. The method may further include adjusting a pixel color withinany of the regions wherein red eye artifact is determined and outputtingan adjusted image to a printer. The pixel color correcting may also beperformed within the printer. The method may further include adjusting apixel color within any of the regions wherein red eye artifact isdetermined and outputting an adjusted image.

A digital apparatus having no photographic film is also provided. Theapparatus includes a source of light for providing illumination duringimage capturing; a digital image capturing apparatus; at least one of animage display and an image output; and a red-eye filter for modifyingpixels indicative of a red-eye phenomenon within the at least one of theimage display and the image output.

The apparatus may further include memory for recording the image afterapplying the filter module for modifying pixels as a modified image. Themodified pixels may be stored directly in the image by replacing thepixels within the image indicative of red-eye phenomenon to create themodified image. The modified pixels may be stored as an overlay of theimage thus preserving the original image. The modified pixels may beprocessed by an external device. The external device may be a personalcomputer and/or a printer.

The apparatus may further include an image output for downloading anintegral image display for printing the image modified by the red-eyefilter. The red-eye correction module may generate an overlay for thepixels indicative of the red-eye phenomenon of the captured image forthe at least one of image display and image output.

The pixels indicative of the red-eye phenomenon may have a color andshape indicative of the red-eye phenomenon and the image may be modifiedto change the color to a black color. Also, the source of light mayselectively provide illumination during image capturing, and the red-eyefilter may be enabled to modify the image in response to the source oflight providing illumination during image capturing. The apparatus mayinclude an exposure control means for determining if the image wascaptured in a condition conducive to the red-eye phenomenon and forgenerating a red-eye signal in response thereto. The red-eye filter maybe further enabled in response to the red-eye signal.

The red-eye filter may further include a false-detection avoidanceapparatus which enables modification of the pixels indicative of thered-eye phenomenon in response to an absence of color indicative of thered-eye phenomenon with in a vicinity of and exclusive to the pixels.The red-eye filter may further include a false-detection avoidanceapparatus which enables modification of the pixels in response to one ormore of a substantially white colored region, an iris ring and aneyebrow line within a vicinity of the pixels. The red-eye filter maydetect the pixels within the image indicative of a red-eye phenomenonbased on one or more of a substantially white colored region, an irisring and an eyebrow line within a vicinity of the area.

The red-eye filter may include a pixel locator for locating the pixelshaving a color indicative of the red-eye phenomenon; a shape analyzerfor determining if a grouping of at least a portion of the pixelslocated by the pixel locator include a shape indicative of the red-eyephenomenon; and a pixel modifier for modifying the color of the pixelswithin the grouping. The false-detection analyzer may further processthe image in a vicinity of the grouping for details indicative of aneye, and for enabling the pixel modifier in response thereto. Theapparatus may further include an exposure analyzer for determining ifthe image was recorded in a condition indicative of the red-eyephenomenon. The red-eye filter may further include an exposure analyzerfor determining if the image was recorded in a condition indicative ofthe red-eye phenomenon.

The exposure analyzer may determine if the image was recorded in acondition indicative of the red-eye phenomenon including determiningwhether the light source was used during image recording. The exposureanalyzer may determine if the image was recorded in a conditionindicative of the red-eye phenomenon including determining whether lowambient lighting conditions existed during image recording. The exposureanalyzer may determine if the image was recorded in a conditionindicative of the red-eye phenomenon. The exposure analyzer may furtherinclude a distance analyzer for determining if the subject was at arelatively close distance to the apparatus during image recording.

A portable digital image acquisition apparatus having no photographicfilm is also provided. The apparatus includes an integral flash forproviding illumination during image recording; a digital image capturingapparatus for recording an image; and a red-eye filter for modifying anarea within the image indicative of a red-eye phenomenon.

The apparatus may further include an integral image display fordisplaying the modified image. The area may have a color and shapeindicative of the red-eye phenomenon and the image may be modified tochange the color to a black color. The integral flash may selectivelyprovide illumination during image recording, and the red-eye filter maybe enabled to modify the image in response to the integral flashproviding illumination during image recording.

The apparatus may include an exposure control means for determining ifthe image was recorded in a condition conducive to the red-eyephenomenon and for generating a red-eye signal in response thereto. Thered-eye filter may be further enabled in response to the red-eye signal.

The red-eye filter may further include a falsing avoidance apparatuswhich enables modification of the area in response to an absence ofcolor indicative of the red-eye phenomenon within a vicinity of andexclusive to the area. The red-eye filter may further include a falsingavoidance apparatus which enables modification of the area in responseto a substantially white colored region within a vicinity of the area.

The red-eye filter may include a pixel locator for locating pixelshaving a color indicative of the red-eye phenomenon; a shape analyzerfor determining if a grouping of at least a portion of the pixelslocated by the pixel locator comprise a shape indicative of the red-eyephenomenon; and a pixel modifier for modifying the color of the pixelswithin the grouping. The red-eye filter may further include a falsinganalyzer for further processing the image in a vicinity of the groupingfor details indicative of an eye, and for enabling the pixel modifier inresponse thereto. The red-eye filter may further include an exposureanalyzer for determining if the image was recorded in a conditionindicative of the red-eye phenomenon.

A method of filtering a red-eye phenomenon from an acquired digitalimage comprising a multiplicity of pixels indicative of color, thepixels forming various shapes of the image, is further provided. Themethod includes analyzing meta-data information including informationdescribing conditions under which the image was acquired, digitizedand/or captured; determining one or more regions within the digitalimage suspected as including red eye artifact; and determining, based atleast in part on the meta-data analysis, whether the regions are actualred eye artifact.

The method may further include obtaining anthropometrical information ofhuman faces and the determining, based at least in part on saidmeta-data analysis, whether the regions are actual red eye artifact, maybe based further on the anthropometrical information. The filteringmethod may be executed within a portable image acquisition device,having no photographic film. The filtering method may be executed as apost-processing step on an external computation device. The meta-datainformation describing the conditions under which the image was acquiredmay include an indication of whether a flash was used when the image wasacquired. The determining whether the regions are actual red eyeartifact may be performed as a probability determination process basedupon multiple criteria. The method may include adjusting a pixel colorwithin any of the regions wherein red eye artifact is determined andoutputting an adjusted image to a printer. The pixel color correctionmay also be performed within the printer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram of an acquisition device operating inaccordance with a preferred embodiment.

FIG. 2 illustrates a high level workflow of detecting red eye artifactsin digital images in accordance with a preferred embodiment.

FIGS. 3 a-3 d schematically depicts a light sensor, and the formation ofa digital pixelated image on it, in accordance with a preferredembodiment.

FIG. 4 describes a process of collecting, forwarding and analyzingmeta-data as part of red-eye detection in accordance with a preferredembodiment.

FIG. 5 illustrates by means of geometric optics, a relationship betweenan object and an image based on a distance to the object and the focallength, where the focal length is the distance from the image principalplane of the optical system to the image focal plane, which is the planewhere the image of the object situated at infinity is formed.

FIG. 6 illustrates a relationship between focal length of a lens anddepth of field, and an object size as it appears on an image.

FIGS. 7 a-7 c illustrate some anthropometric measurements of a humanface for an adult male and female.

FIGS. 8 a-8 b show a workflow diagram describing a statistical analysisof an image using anthropometric data in accordance with a preferredembodiment.

FIG. 9 depicts a spectral response of an acquisition system based onspectral sensitivity curves of a hypothetical three color sensor, thespectral distribution of a generic light source and the spectralcharacteristics of a object being photographed, in accordance with apreferred embodiment.

INCORPORATION BY REFERENCE

What follows is a cite list of references which are, in addition tothose references cited above and below herein, and including that whichis described as background, the invention summary, brief description ofthe drawings, the drawings and the abstract, hereby incorporated byreference into the detailed description of the preferred embodimentsbelow, as disclosing alternative embodiments of elements or features ofthe preferred embodiments not otherwise set forth in detail below. Asingle one or a combination of two or more of these references may beconsulted to obtain a variation of the preferred embodiments describedin the detailed description below. Further patent, patent applicationand non-patent references are cited in the written description and arealso incorporated by reference into the preferred embodiment with thesame effect as just described with respect to the following references:

U.S. Pat. Nos. 4,285,588, 5,016,107, 5,070,355, 5,202,720, 5,537,516,5,452,048, 5,748,764, 5,761,550, 5,781,650, 5,862,217, 5,862,218,5,991,549, 6,006,039, 6,433,818, 6,510,520, 6,516,154, 6,505,003,6,501,911, 6,496,655, 6,429,924, 6,252,976, 6,278,491;

United States published applications no. 2003/0058349, 2003/0044177,2003/0044178, 2003/0044070, 2003/0044063, 2003/0025811, 2002/0150306,2002/0041329, 2002/0141661, and 2002/0159630;

PCT published applications no. WO 03/026278, WO 99/17254; and WO01/71421; and

Japanese patents no. JP 04-192681, JP 2000/134486, and JP 2002/271808;and

European patents no. EP 0 884 694 A1, EP 0 911 759 A2,3, EP 1 293 933A1, EP 1 199 672 A2, EP 1 288 858 A1, EP 1 288 859 A1, and EP 1 288 860A1; and

Matthew Gaubatz, et al., “Automatic Red-eye Detection and correction”,IEEE ICIP, 2002, pp. I-804-I-807.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments described below include methods for detecting redeye artifacts in digital images. Methods are also described forutilizing meta-data gathered as part of the image acquisition to removesuch red-eye artifacts. In addition, methods for enhancing the accuracyof detection of red eye artifacts based on a-priori knowledge of thecamera sensor, the acquisition mechanism and the color transformationare described. Methods are described for enhancing the speed ofdetection of red eye artifacts in digital images, and for reducing theamount of false detection of regions suspected to be red-eye artifacts.A method for user-selected tradeoff between the reduction of falsedetection and the improvement of positive detection is also described.In addition, a way to estimate the size of faces is provided, and inparticular the eyes in an image and in particular the size of eyes infaces based on the acquisition data. A way to improve the detection ofthe eyes based on anthropometric analysis of the image is also provided.An improvement is described for the detection of the eyes based on aBayesian statistical approach. An improvement is also described for thedetection of the red eye artifacts based a priori knowledge of the filmmanufacturer, the film type and/or the emulsion batch of the film. Animprovement is also described for the detection of the eye artifactbased on a priori knowledge of the scanner its light source and thecolor sensors of the scanner.

In one embodiment, a digital camera has a built in flash, an imageacquisition mechanism and a way to save the acquired data. The methodsof the preferred embodiments are generally applicable to digital imageacquisition devices, such as digital cameras and scanners, and to andoutput devices such as printers and electronic storage devices. When theterms digital camera and output device or printer are used, it isgenerally meant to more broadly, respectively include digital imageacquisition devices and digital data output devices.

A printer that receives image data from an original image acquisitiondevice such as a digital camera or scanner may include a display thatshows the image, or may be configurable to be cable, rf, or otherwiseconnected to a display. In this way, the image may be previewed beforeprinting, and if desired, corrected and previewed again until the imageis as desired for printing. Another alternative is to permit the imageto be printed as a thumbnail with a preview of the red eye correctedregions to save on printing time and money. These regions of interest asto red eye correction may be circled or otherwise indicated in theprinter viewer and or in the printed thumbnail. In a case where thecamera itself includes the red eye correction software, firmware, and/ormemory or other electronic component circuitry, then such preview and/orthumbnail capability may be included within the camera that may itselfbe cable, rf, network and/or otherwise connected to the printer.

The digital camera or other acquisition device preferably has thecapability of analyzing and processing images. Alternatively, theprocessing of the images can be done outside of the camera on a generalpurpose or specialized computer after downloading the images or on adevice that is acting as a hosting platform for the digital camera. Sucha device may be, but is not limited to, a hand held PC, a print server,a printer with built in processing capability, or cell phone equippedwith a digital camera. Alternatively the acquisition process can be ofan analog image, such as scanning of a film based negative or reversalfilm, or scanning of a photographic print.

The accuracy of a detection process may be measured by two parameters.The former is the correct detection, which relates to the percentage ofobjects correctly detected. The second parameter for evaluatingsuccessful detection is the amount of mis-classifications, which is alsodefined as false detections or beta-error. False detections relate tothe objects falsely determined to have the specific characteristics,which they do not possess.

Overall, the goal of a successful detection process is to improve theaccuracy of correct detections while minimizing the percentage of falsedetections. In many cases there is a tradeoff between the two. When thesearch criterion is relaxed, more images are detected but at the sametime, more false detections are typically introduced, and vice versa.

In order to improve the accuracy of the red eye detection andcorrection, a preferred embodiment utilizes a priori information aboutthe camera or camera-specific information, anthropometric informationabout the subject, and information gathered as part of the acquisitionprocess. That is, although information gathered as part of theacquisition process may relate to the camera or other digitalacquisition device used, information relating to those parameters thatare adjustable or that may change from exposure to exposure, based onuser input or otherwise, are generally included herein as informationrelating to the acquisition process. A priori or camera-specificinformation is camera-dependent rather than exposure-dependent. Forexample, a-priori information about the camera may include any of thecolor sensitivity, spectral response or size of the camera sensor,whether the sensor is CCD or CMOS, and color transformations from theRAW data gathered by the sensor, e.g., CCD, to a known color space suchas RGB, the f-stop, or other camera-specific parameters understood bythose skilled in the art, or combinations thereof. In the case ofscanning such a-priori information may include the color sensitivitycurve of the film, the color sensitivity of the scanner sensor, whetherCCD or CMOS, whether linear or area sensors, the color transformationsfrom the RAW data gathered by the scanner to a known color space such asRGB. Acquisition data may include any of the focal distance asdetermined by the auto focus mechanism of the digital camera, the powerof the flash including whether a flash was used at all, the focal lengthof the lens at acquisition time, the size of the CCD, the depth of fieldor the lens aperture, exposure duration, or other acquisition parametersunderstood by those skilled in the art, or combinations thereof.Anthropometric data may include first and higher order statistics, whichis an average and a variability of an expected size and ratio betweendifferent parts of the human body, and particularly the facial region.

Based on utilizing the aforementioned information, preferred embodimentsdescribed herein achieve a more accurate detection of the regionscontaining red eye artifacts. Based on this detection, the processor,whether in the camera or on a different device, can perform a correctionstep.

FIG. 1 is a components diagram in accordance with a preferredembodiment. Block 100 describes the image acquisition device which canbe a digital camera in different packaging such as a digital stillcamera, a lens connected to a hand held computer, a cell phone withimage capturing capability, a video camera with still image capturingcapability, etc.

In the image capture apparatus 100, there are a few components shown inblock form in FIG. 1. The first is the light sensor 102 that can be aCCD, CMOS or any other object that transforms light information intoelectronic encoding. Most cameras are equipped with a built in flash104, also referred to as a strobe. In many cases, the camera strobe isphysically close to the lens, which tends to accentuate the occurrenceand strength of the red eye artifact. In addition, the camera isequipped with a lens 106. The relevant parameters of the lens duringacquisition include the aperture 114, or a f-stop, which primarilydetermines the depth of field, the focal length 112 which determines theenlargement of the image, and the focusing distance 116 which determinesthe distance to the objects that the lens 106 was focused at.

Block 130 of FIG. 1 describes the red eye filter that performs a processof detection and correction of the red eye artifacts in accordance witha preferred embodiment. The process can be done in the camera as part ofthe acquisition stage, in the camera at a post processing stage, duringthe transferring of the images from the camera to an external devicesuch as a personal computer, or on the external device as a postprocessing stage, such as in the image transfer software or imageediting software.

The red eye filter includes two main stages. Block 132 describes ameta-data analysis module 132, where the image and the probability forred eye artifacts are evaluated based on the acquisition data and/orother meta-data. Block 138 describes the pixel-based analysis where theimage data is used. The pixel-based analysis 138 preferably receivesinformation from the meta-data stage 132. Therefore, the decision on thepixel level may vary based on the conditions under which the image wascaptured and/or other meta-data. Block 160 describes the image storagecomponent 160 that saves the image after the red eye correctionoperation.

FIG. 2 is a workflow representation corresponding to the preferredcamera embodiment illustrated at FIG. 1. The image capture stage isdescribed in block 200. This operation includes the pre-acquisitionsetup 210, where the user and/or the camera determine preferred settingssuch as f-stop 212, flash on/off 214 and/or focal length 216. The imagecapture stage 200 also includes acquisition or picture taking 226, andtemporary storage in block 228 in its final form or in a raw form thatcorresponds to the image as captured by the light sensor 102 of FIG. 1.As part of the capture process, the camera determines the bestacquisition parameters in the pre-acquisition stage 210. Such parametersmay include the right exposure, including gain, white balance and colortransformation, and in particular aperture settings 212 and whether touse flash 214. In addition, the user may decide on the focal length 216of the lens 106, which is also be referred to as the zoom position.

The image after being stored in block 228, is then processed for red eye230 in accordance with a preferred embodiment, among other stages ofprocessing that may include color corrections, compression, sharpening,etc. The red eye filter preferably includes two main operations. The redeye detection 240 and red eye correction 250.

The red eye detection 240 includes a first stage of analyzing theperipheral or external data, or meta-data 242, a stage of transferringthe revised data 244, and the specific red eye detection 246, based onpixel analysis.

The red eye correction is illustrated at FIG. 2 as the operation 250where any image modifications based on the results of the detectionstage 240, are applied to the image. At this stage 250, correction maybe burned into the data 252, thus replacing the damaged pixels, saved asa list of the pixels that need to be changed with their new value in theheader of the image or externally 254, and/or presented to the user 256,requesting the user to take an action in order to apply the corrections,or a combination of these operations. The image, with the correctionsapplied as described in 240, is then preferably saved in block 260.

FIGS. 3 a-3 d illustrates in detail the image as created on the receptor102 of FIG. 1, which is located at the image plane of the opticalsystem. Such receptor can be any electro-photosensitive object such asCCD or CMOS.

FIG. 3 a illustrates a grid type CCD. Each one of the smaller squares(as illustrated by block 302) is a cell, which is sensitive to light.The CCD size 304 is calculated as the diagonal of the rectangle made ofWidth 306 and Height 308.

FIG. 3 b illustrates how a face may be projected onto the CCD. FIG. 3 cillustrates how the image is pixelized, where the continuous image istransformed into a grid based image.

FIG. 3 d is more specific to the image as created by a human eye. Theimage of the eye will include the iris 342 as well as the pupil 344,which is usually the locations where red-eye artifacts occur. The whitepart 346 of the eye is also a component of the human eye illustrated atFIG. 3 d and which can be used in red-eye detection, particularlyfalse-detection avoidance.

FIG. 4 illustrates various meta-data information that can be utilized aspart of a preferred embodiment as a priori input, and the potentialoutcome of such data analysis. For example, blocks 412, 422, and 432illustrate an operation of red-eye detection relating to the use ornon-use of flash. The information whether the flash is used or not,Block 412, is forwarded at operation 422 to red-eye pre-processing 432to determine whether there is reason to launch the red-eye filter. If aFlash, as determined in 412 is not used, there is preferably no reasonto apply the redeye filter. This is a reasonable estimation for consumerlever cameras where most of the red eye is created, as described in theintroduction, by the small disparity between the strobe unit and thelens.

Blocks 414, 424, 434 describe a collection of acquisition meta-data,wherein non-exhaustive examples are provided including the distance tothe object, the aperture, CCD size, focal length of the lens and thedepth of field. This data is usually recorded on or with the image atacquisition. Based on this information, as transferred to the filter atoperation 424, the filter can determine at operation 434, e.g., a rangeof potential sizes of red eye regions.

Blocks 416, 426, 436 relate to specific information that is unique tothe camera. The color composition, e.g., of the image is determined by afew parameters which include the CCD response curves as illustrated inFIG. 9 (see below), and the potential color transformations from therecorded, raw image data such as color correction, gain adjustment andwhite balance to a known color space such as RGB or YCC. Suchtransformations can be presented in the form of lookup tables,transformation matrices, color profiles, etc.

Based on the knowledge of the transfer from operation 426, the softwarecan better determine a more precise range of colors at operation 436that are good candidates for the red eye artifacts. This information canadvantageously narrow down the potential red eye regions based on thevariability of sensors and color correction algorithms. It may also helpto eliminate colors that, without this knowledge, could be falselyidentified as potential red eye region candidates, but are not such incase of a specific combination of sensor and color transformation.

FIG. 5 depicts illustrative information that can be gathered todetermine the relative size of the object. The ratio of the image sizedivided by image distance, and the object size divided by the objectdistance, are approximately equal, wherein the image size divided by theobject size is defined as the magnification of the lens 106. If oneknows three out of the four values, namely focal length 112, distance toobject 116, and object size 516, one can estimate the size of theobject:

$\frac{{Object}\mspace{14mu} {size}\mspace{14mu} (516)}{{distance}\mspace{14mu} {to}\mspace{14mu} {object}\mspace{14mu} (116)} = \frac{{image}\mspace{14mu} {size}\mspace{14mu} (512)}{{focal}\mspace{14mu} {length}\mspace{14mu} (112)}$

If one knows three out of the four values, namely focal length112,distance to object 116, and object size 516 one can estimate theimage size 512:

${{Object}\mspace{14mu} {size}\mspace{14mu} (516)} = \frac{{distance}\mspace{14mu} {to}\mspace{14mu} {object}\mspace{14mu} {(116) \cdot {image}}\mspace{14mu} {size}\mspace{14mu} (512)}{{focal}\mspace{14mu} {length}\mspace{14mu} (112)}$

However, the parameter values described above are usually not knownprecisely. Instead, distributions of values can be estimated based ondifferent reasons as depicted in FIGS. 6, 7 and 8.

FIG. 6, illustrates the variability generated by the depth of field.Depth of field is defined as the range of distances from the camera tothe objects where the images of the objects are captured sufficientlysharp. For a fixed length lens, the depth of field is a function of theaperture. The more open the aperture is, the shallower the depth offield is.

As can be seen in FIG. 6, due to the fact that the depth of field can berather large, the distance to the objects still in focus can vary.Therefore the parameter

Distance_to_Subject

is rather a range:

Distance_to_Subject_(Close) _(—)_(range)≦Subject≦Distance_to_Subject_(Far) _(—) _(range)

The reason why this information is important and has to be taken intoconsideration is depicted in FIG. 6. In this case, two objects, a tree614 and a house 624 are located in close distance 616, and further away626 respectively. Even though the tree, 614 and the house 634 are thesame size, the sizes of the objects or the projections of the objects onthe image plane are different and the tree image, 636 being closer tothe camera appears much larger than the house 646.

FIG. 7 includes some relevant anthropometrical values for male andfemale averages. FIG. 7-a is an average male and FIG. 7-b is an averageadult female. For example, for adult male, 700, the distance between theeyes, 714, is on average 2.36″, the distance between the eyes and thenostrils, 724, is 1.5″ the width of the head, 712 is 6.1″ etc.

However, this is only the first order approximation. There is a secondorder approximation, which is the overall variability of the values.Such variability once again needs to be calculated into the formula.

Or:

Subject_Size_(Small)≦Subject_Size≦Subject_Size_(Large)

The object size, in order to be considered as a candidate for being aface, and eye or any known object will be:

$\quad\begin{matrix}{\frac{{Subject\_ SizeSmall}*{Focal\_ Length}}{{Distance\_ To}{\_ Object}_{{Far}\_ {Range}}} \leq {Object\_ Size} \leq} \\{\mspace{416mu} \frac{{Subject\_ Size}_{large}*{Focal\_ Length}}{{Distance\_ To}{\_ Object}_{{Close}\_ {Range}}}}\end{matrix}$

Specifically, as seen in FIG. 7-c, the average size of an eyeball, 770,is roughly 1″, or 24 mm, and the average size of the iris, 772, is halfin diameter to the full eye, or 0.5″ or 12 mm in diameter. The pupil,774 can be as small as a few millimeters, and dilated to as large as thesize of the iris. Fortunately, in the case of red-eye artifacts, whichhappen primarily in low lighting conditions that required a flash, thepupil will be on the dilated side.

The variability in this case is not only for different individuals, butalso variability based on age. Luckily, in the case of eyes, the size ofthe eye is relatively constant as the person grows from a baby into anadult, this is the reason of the striking effect of “big eyes” that isseen in babies and young children. The average infant's eyeball measuresapproximately 19½ millimeters from front to back, and as describedabove, grows to 24 millimeters on average during the person's lifetime.Based on this data, in case of eye detection, the size of the objectwhich is the pupil which is part of the iris, is limited, when allowingsome variability to be:

9 mm≦Size_Of_Iris≦13 mm

The object size as calculated above is going to be in actual physicalsize such as millimeters or inches. For this invention to become useful,this information needs to be presented measured in pixel sizes.

Returning to FIG. 3 a, the size of the sensor is depicted by 304, whichis the diagonal of the sensor. Based on that, and the ratio between thewidth, 306 and the height, 308, the width and height can be calculatedas a Pythagorean triangle.

Sensor_Diagonal_Size=√{square root over (width²+Height²)}

Knowing the sensor resolution, the size of object can now be translatedinto pixel size.

For Example:

-   Given a ½ inch (12 mm) CCD, with an aspect ratio of 2:3, and a    2,000×3,000 CCD resolution:-   The width of the CCD is:

${12\mspace{14mu} {mm}} = {\sqrt{\left( {2\alpha} \right)^{2} + \left( {3\alpha} \right)^{2}} = {{{\sqrt{13}\alpha}\therefore {3\alpha}} = {{3 \times {12/\sqrt{13}}} \approx {3 \times 3.3} \approx {10\mspace{14mu} {mm}}}}}$

and therefore, for a 3000 pixel width, a 1 mm object size is equal toroughly 300 pixels.

Or

Image_Size_(in) _(—) _(pixels)=Image_Size_(in) _(—) _(millimeters)

Based on this formula, when an image is now detected, its size in pixelsis compared to the range allowed, and decided whether the object is acandidate or not.

An example is depicted in FIG. 3 d where a hypothetical eye is displayedin pixels, and in this case, the iris 342, is roughly 11 pixels, and thepupil, 344, 6 pixels in diameter. With the added knowledge of thedistance to the object and the focal length of the lens, this inventionpresents a decision process capable of rejecting the objects, 346 thatare not eyes and selecting most likely candidates to be an eye based onthe sizes of the captured images of the objects.

FIG. 8 describes a preferred workflow to perform, the analysis based onthe sizes of objects, and in the case of human beings, theanthropometrical analysis. The input is the acquisition data 434, asdescribed in FIG. 4, and human anthropometric data, 800 as depicted inFIGS. 7 a and 7 b.

Step 810 describes the calculation of potential size and distribution ofthe objects, as corresponds to the camera resolution. This process wasfully defined above. Note that this calculation can be done on the flyor alternatively pre-calculated values can be stored in a database tospeed up the processing.

When looking for eyes in an image, but not limited specifically to eyes,given regions suspected as eyes, 820, a preferred embodiment proposes tocheck, 830 whether the regions fall within the size and distribution ascalculated above in 820. If the size is too large or too small, thesystem can determine, 890 that the probability for this object to be aneye is low. However, this is a probabilistic result and not necessarilya conclusive one. In other words, the specific region 820 has now lowprobability assigned to it as a potential eye. If the region is fallinginside the allowed size, the probability, 880 are raised.

This preferred embodiment describes additional steps to refine thedecision, or increase the probability, by analyzing additional cluessuch as the existence of a second eye, 832, the surrounding facialfeatures, 834 such as the overall shape of the face, the hair, necketc., the existence of lips in proximity to the eyes, 836, the nostrils838 etc.

In each step, the question asked is whether the new feature is part ofthe region, 840. If the reply is positive, then the probability foridentifying the area as an eye is raised, 850, and if negative, theprobability is reduced, 860. Of course, this probabilistic approach canbe useful to create a better set of criteria in deciding whether thedetected object is what the system is looking for. In more detail, thedetection process involves two types of allowed errors also known asType-I and Type-II errors, or also referred to as α-error, which is theacceptable probability of making a wrong decision, or a false positiveand β-error, which is the acceptable probability of not detecting atall. Based on this approach, the probability as decreased or increasedin steps 850 and 860 are always compared against the two criteria α andβ.

Alternatively to the classical statistical approach, this analysis canbe done using Bayesian approach. As defined above, Bayesian probabilitycan be calculated based on:

${P\left( {B_{i}A} \right)} = \frac{{L\left( {AB_{i}} \right)}{P(B)}}{\sum\limits_{{all} - j}^{\;}\; {{L\left( {AB_{j}} \right)}{P\left( B_{j} \right)}}}$

This is further depicted in FIG. 8 b. Specifically to this embodiment,the events are:

-   A=Region detected is red eye, as depicted in Block 870-   B_(j)=the various detected features as defined in blocks 872,874,876    and 878, 834,836 and 838.-   A∩B_(j)=Probability that the area is red eye AND that another    attribute is found. For example If B_(i) is the probability of    detecting lips,-   A∩B_(j) is the probability that the region is an eye and that the    lips are detected.-   P(B_(i)|A) is the probability that lips exist when eye is detected.-   And-   P(A|B_(i)) is the probability of eye detection given the probability    of lips detection.

FIG. 9 illustrates a different kind of information that can be veryuseful in determining the existence of red eye artifacts, using thecolor sensitivity of the capturing system such as a digital camera.Alternatively the capturing system may be analog capture such as filmfollowed by a digitization process such as scanning

The graph in FIG. 9 describes the relative response, 950 as a functionof the visual wavelength 910, of the three sensors for blue, 932, Green934, and Red 936, of a typical CCD type sensor. Similar graph, althoughwith different response curve describes the response of the differentlayers for photographic film.

The x-axis, which is the wavelength range of the human visual system, isexpanded to include infrared and ultraviolet, which may not be visibleto the human eye but may record on a sensor. The y-axis is depicted inrelative value as opposed to an absolute one. The three Red, Green, andBlue spectral response functions as functions of the wavelength aredefined respectively as:

R(λ), G(λ), B(λ)

Given a light source 940 defined as a spectral response curve L(λ), thelight source 940 when reaching the three different color sensors, orcolor pigments on film will generate a response for each of the colorsas defined mathematically as the integral of the scalar multiplicationof the curves. The range of integration is from the low wavelengthregion UV to the highest IR.

R = ∫_(λ − UV)^(λ − IR)R_(λ) × L_(λ) λ, G = ∫_(λ − UV)^(λ − IR)G_(λ) × L_(λ) λB = ∫_(λ − UV)^(λ − IR)B_(λ) × L_(λ) λ

to create a tristimulus value of {R, G, B}

Those skilled in the art are familiar with the fact that differentspectral responses may create the same tristimulus values due to thescalar reduction from a 2 dimensional representation to a single value.This effect is also known as Metamerizm which can be a property of thesensor's/film's metamerizm, the human visual system metamerizm, or thelight source's metamerizm.

Due to the many variable parameters, it is relatively hard to find aspecific color that can be a fixed-reference-point in an image. Thereason is that the reflected colors are usually dependent on manyfactors and especially on the ambient light. However, Red Eye artifacts,as previously explained, are results of the reflection of the strobelight, which has very well defined characteristics, from the vascularmembrane behind the retina, which is rich in blood vessels. In mostcases, the effect of the external ambient light is relatively low, andthe red-eye effect can be considered as a self-illuminating object, withmore precise spectral characteristics than other objects. An example ofsuch spectral response, which is a combination, of the flash spectralresponse, which is relatively broad and the blood vessels inside theeye, is depicted in block 940.

Given the spectral sensitivity of the sensor:

R(λ), G(λ), B(λ)

and the reflection of the flash light in the eye, as defined by 950,E(λ), the red eye tristimulus values for this specific sensor are:

{R, G, B}_(red − eye) = ∫_(λ − UV)^(λ − IR){R, G, B}_(λ) × L_(λ) λ

This value of {R, G B}_(red-eye) is relatively constant for a givencamera. However, due to the difference in the response between differentsensors, these values are not constant across different cameras.However, with the knowledge of the response curves above, one candetermine a much closer approximation of the range or red colors basedon this information. Note that it is not only the value of the Red thatmay help in such determination, but also the residual response of thered eye on the Green and even less the blue sensor. One skilled in theart knows that most cameras perform additional transformations forexposure and tone reproduction for images before saving them intopersistent storage. An example of such transformation will be aconcatenation of color correction and tone reproduction as a function ofthe pixel value:

Given a Raw pixel value of:

{R, G, B}_(RAW-CCD)

as transformed via three lookup tables. For example for red lookuptable:

R−LUT (Raw−Pix): {input_values}→{output_values}

For example the Red lookup table R-Lut can be a gamma function from 10bit raw data to 8 bits as follows:

R_(LUT)(Raw−Pix): {0..1024}→{0..256}

R _(LUT)(x)=(R _(RAW-CCD)/1024)²²*256

and the inverse function

R ⁻¹ _(LUT)(x)=(R _(LUT) _(—) _(RAW)/256)^(1/2.2)*1024

the {R,G,B} values after transformed through the lookup table will be:

$\left\{ {R,G,B} \right\}_{{LUT} - {RAW}} = \begin{Bmatrix}{{R_{LUT}\left( R_{{RAW} - {CCD}} \right)},{G_{LUT}\left( G_{{RAW} - {CCD}} \right)},} \\{B_{LUT}\left( B_{{RAW} - {CCD}} \right)}\end{Bmatrix}$$\left\{ {R,G,B} \right\}_{new} = {\left\{ {R,G,B} \right\}_{{LUT} - {RAW}} \times \begin{bmatrix}{RR} & {RG} & {RB} \\{GR} & {GG} & {GB} \\{BR} & {BG} & {BB}\end{bmatrix}}$

With the internal knowledge of these transformations, one can reversethe process, to reach the RAW values as defined above.

$\left\{ {R,G,B} \right\}_{{LUT} - {RAW}} = {\begin{bmatrix}{RR} & {RG} & {RB} \\{GR} & {GG} & {GB} \\{BR} & {BG} & {BB}\end{bmatrix}^{- 1} \times \left\{ {R,G,B} \right\}_{NEW}^{T}}$ and$\left\{ {R,G,B} \right\}_{RAW} = \begin{Bmatrix}{{R_{LUT}^{- 1}\left( R_{{LUT} - {RAW}} \right)},{G_{LUT}^{- 1}\left( G_{{lut} - {raw}} \right)},} \\{B_{LUT}^{- 1}\left( B_{{LUT} - {RAW}} \right)}\end{Bmatrix}$

and the value of the raw tristimulus values can be then determined andused for the exact matching. Similar transformations are performed bydigital scanners in order to correct for sub optimal images such asunderexposure, or wrong ambient light. Reversing the process may bedifficult in its pure mathematical sense e.g. the conversion functionmay through the transformation not be fully reversible. Such issuesoccur for example when the pixel values are clipped or condensed. Insuch cases, there is a need to define a numerical approximation to theinverse function.

The preferred embodiments described above may be modified by adding orchanging operations, steps and/or components in many ways to produceadvantageous alternative embodiments. For example, there are generallytwo approaches to removing red-eye from images. The traditional oneincludes an attempt to reduce one or more reasons that cause red eyeprior to taking the picture. The second approach is the post processingof the images to detect and then eliminate the red-eye artifact in apost processing stage, as described in accordance with a preferredembodiment.

There are many ways that analysis processes operating within a cameraprior to invoking a pre-flash may be configured. Various conditions maybe monitored prior to the photograph and even before the pre-flash isgenerated. These conditions may include the ambient light level and thedistance of the subject from the camera (see, e.g., U.S. Pat. No.5,070,355 to Inoue et al., hereby incorporated by reference). Accordingto one embodiment, steps may be taken that generally reduce theoccurrences of a pre-flash that may otherwise be used when warranted. Inanother embodiment, the use of pre-flash is eliminated altogether. Inthis embodiment, the red-eye phenomenon in a miniature camera with anintegral strobe or flash is eliminated and/or prevented without using apre-flash, preferably through post-processing, red-eye eliminationprocedures as described above.

The use of meta-data for the post-processing of digital images has beendescribed above in accordance with a preferred embodiment (see also USPubl. Pat. App. No. 2003/0058349 to Takemoto). Meta-data contained in adigital image may be analyzed, as may be referred to as EXIF tags, orsimply tags, and utilizing such information, global post-processing maybe performed on the image to adjust the image tone, sharpness and/orcolor balance. Another way to use meta-data is in the photo-finishingindustry, where a digital image may be post-processed to optimize theoutput from a printing system. Examples of this use of meta-data areprovided at U.S. Pat. Nos. 6,505,003 6,501,911 and 6,496,655 to MalloryDesormeaux, hereby incorporated by reference. A hybrid camera may beused which saves a copy of the original image containing meta-data andimplements a scheme which allows control over saving the imagecontaining metadata outside the camera. Image meta-data may also berecorded onto a standard camera film and the meta-data may besubsequently recovered to assist in the post-processing of the film (seeU.S. Pat. No. 6,429,924 to Milch, hereby incorporated by reference).Advantageously in accordance with a preferred embodiment, imagemeta-data may be used to determine a size range of objects and relatedfeatures within an image, in addition to the correction of globalparameters such as image tone, sharpness and color balance.

A red-eye correction procedure may begin with detecting a human face ina digital image and, based on this detection, finding the eyes in theface (see, e.g., U.S. Pat. No. 6,252,976 to Schildkraut and Gray, U.S.Publ. Pat. App. No. 2003/0044070 to Fuersich et al., and U.S. Pat. No.6,278,491 to Wang and Zhang, which are incorporated by reference). Thisprocedure may preferably begin with detecting one or more face regionsof a person or persons in a digital image, followed by detecting an eyeregion or eye regions in each face, and finally determining if red-eyedefects exist in the subject's eyes. In the '976 patent, a complexprocedure is described for detecting faces and balanced eye-pairs from askin-map of the image. This task involves several partitioning andre-scaling operations. Significant additional processing of a potentialface region of the image then follows in order to determine if amatching pair of eyes is present. Finally, the image pixels in thedetected eye regions go through a complex scoring process to determineif a red-eye defect is present.

In a preferred process, a simplified and thus generally less resourceintensive, image processing technique is used relative to thosedescribed at the '976 and '491 patents which detect face and eye regionsin an image and subsequently verify the presence of red-eye defects. Anadvantageous technique will preferably not weight too heavily upondetecting balanced eye pairs, as this approach can get complex andresource intensive when two or more facial regions overlap or are inclose proximity to one another in a digital image. According to apreferred embodiment herein, metadata is used to simplify the detectionof red-eye defects in a digital image. For example, one or moreexclusion criteria may be employed to determine that no flash was used(see also U.S. Publ. Pat. App. No. 2003/0044063 to Meckes et al.).

A range of alternative techniques may be employed to detect and verifythe existence of red-eye defects in an image (see, e.g., U.S. Publ. Pat.Apps. No. 2003/0044177 and 2003/0044178 to Oberhardt et al., herebyincorporated by reference). A camera may include software or firmwarefor automatically detecting a red-eye image using a variety of imagecharacteristics such as image brightness, contrast, the presence ofhuman skin and related colors. The analysis of these imagecharacteristics may be utilized, based on certain pre-determinedstatistical thresholds, to decide if red-eye defects exist and if aflash was used to take the original image. This technique may be appliedto images captured on conventional film, which is then digitallyscanned, or to initially digitally-acquired images. Preferably, metadatais used that can be generated by a digital camera or otherwise recordedin or associated with the body of a digital image initially captured orscanned. In accordance with a preferred embodiment, meta-data an/oranthropometric data may be used to validate the existence of a red-eyedefect in an image.

Further techniques may be used alternatively to the preferredembodiments described above for removing flash artifacts from digitalimages. Two copies of a digital image may be captured, one taken withflash illumination and a second taken without flash illumination, andintensity histograms of the two images may be compared in order tolocate regions of the image where flash artifacts occur and correctthese by reducing intensities in these regions (see, e.g., US Publ. Pat.App. No. 2002/0150306 to Baron). Specular reflections may be removed dueto the flash and red-eye can be reduced in this way. However, even Baronrecognizes that the technique may involve the setting of separatethresholds for each of the RGB image colors. A technique such as thiswill generally further involve use of some additional knowledge of thecaptured image if it is to be relied upon for correctly locating andidentifying red-eye defects.

Another technique may involve the identification of small specularreflections that occur in the eye region when flash illumination is used(see, e.g., WO 03/026278 to Jarman, which is hereby incorporated byreference). This procedure may be used to detect red-eye defects withoutfirst detecting a human face or eye region. It is preferred, however, touse camera-specific information, or other image metadata such asacquisition data, or anthropometric data, or a combination thereof, toassist in the confirmation of a red-eye defect.

Digital cameras can also be customized using demographic groups (see,e.g., U.S. Publ. Pat. App. No. 2003/0025811 to Keelan et al., herebyincorporated by reference). The rationale for this technique is thatcertain aspects of image processing and the image acquisition processsuch as color and tone balance may be affected by both age-related andracial factors. It is also noted that both racial and age factors canaffect the level of red-eye defects, which occur, and thus the pre-flashalgorithms and flash-to-lens spacing for a digital camera may beadjusted according to the target market group based on age andnationality. Human faces may be detected and classified according to theage of the subjects (see, e.g., U.S. Pat. No. 5,781,650 to Lobo et al.).A number of image processing techniques may be combined withanthropometric data on facial features to determine an estimate of theage category of a particular facial image. In a preferred embodiment,the facial features and/or eye regions are validated usinganthropometric data within a digital image. The reverse approach mayalso be employed and may involve a probability inference, also known asBayesian Statistics.

The preferred embodiments described herein may involve expanded digitalacquisition technology that inherently involves digital cameras, butthat may be integrated with other devices such as cell-phones equippedwith an acquisition component, toy cameras etc. The digital camera orother image acquisition device of the preferred embodiment has thecapability to record not only image data, but also additional datareferred to as meta-data. The file header of an image file, such asJPEG, TIFF, JPEG-2000, etc., may include capture information such aswhether a flash was used, the distance as recorded by the auto-focusmechanism, the focal length of the lens, the sensor resolution, theshutter and the aperture. The preferred embodiments described hereinserve to improve the detection of red eyes in images, while eliminatingor reducing the occurrence of false positives, and to improve thecorrection of the detected artifacts.

While an exemplary drawing and specific embodiments of the presentinvention have been described and illustrated, it is to be understoodthat that the scope of the present invention is not to be limited to theparticular embodiments discussed. Thus, the embodiments shall beregarded as illustrative rather than restrictive, and it should beunderstood that variations may be made in those embodiments by workersskilled in the arts without departing from the scope of the presentinvention, as set forth in the claims below and structural andfunctional equivalents thereof.

In addition, in methods that may be performed according to preferredembodiments herein and that may have been described above, theoperations have been described in selected typographical sequences.However, the sequences have been selected and so ordered fortypographical convenience and are not intended to imply any particularorder for performing the operations, unless expressly set forth orunderstood by those skilled in the art being necessary.

1. A portable digital apparatus having no photographic film, comprising:(a) a housing; (b) a source of light coupled to the housing forproviding illumination during image capturing; (c) a digital imagecapturing apparatus within the housing; (d) at least one of an imagedisplay and an image output coupled to the housing; and (e) anauto-focus mechanism (f) a red-eye filter within the housing formodifying pixels indicative of a red-eye phenomenon within said at leastone of said image display and said image output, wherein the red eyefilter is configured to modify an area within the image indicative ofred-eye phenomenon based on a metadata analysis, including: (i) acomparison of a calculated range of expected sizes of said area based ondistance information as recorded by the auto-focus mechanism and on oneor more stored human facial anthropometrical values: and (ii) a searchfor a second eye, a surrounding face, lips, nostrils or anotherassociated peripheral region, or combinations thereof, at an estimateddistance based on said distance information as recorded by theauto-focus mechanism.
 2. The apparatus of claim 1, wherein said red-eyefilter comprises: a pixel locator for locating the pixels having a colorindicative of the red-eye phenomenon; a pixel analyzer for determiningif a grouping of at least a portion of the pixels located by said pixellocator comprise one or more parameters indicative of the red-eyephenomenon; and a pixel modifier for modifying the color of the pixelswithin the grouping.
 3. The apparatus of claim 2, further comprising ameta-data analyzer for determining whether to preclude a red-eyephenomenon determination based upon analysis of meta-data.
 4. Theapparatus of claim 3, said meta-data analysis being performed beforesaid pixel analysis.
 5. The apparatus of claim 3, said meta-dataanalyzer comprising an exposure analyzer for determining if the imagewas recorded in a condition indicative of the red-eye phenomenon.
 6. Theapparatus of claim 3, said meta-data comprising digitalapparatus-specific information.
 7. A portable digital apparatus havingno photographic film, comprising: (a) a housing; (b) an integral flashembedded within said housing for providing illumination during imagerecording; (c) a digital image capturing apparatus within said housingfor recording an image; and (d) a red-eye filter within said housing formodifying an area within the image indicative of a red-eye phenomenon,and (e) wherein whether said area within said image is indicative ofsaid red-eye phenomenon is determined based at least in part on ametadata analysis, including: (i) a comparison of a calculated range ofexpected sizes of said area based on distance information as recorded bythe auto-focus mechanism and on one or more stored human facialanthropometrical values: and (ii) a search for a second eye, asurrounding face, lips, nostrils or another associated peripheralregion, or combinations thereof, at an estimated distance based on saiddistance information as recorded by the auto-focus mechanism.
 8. Theapparatus of claim 7, further comprising memory for recording said imageafter applying said red-eye filter, thereby recording a modified image.9. The apparatus of claim 8, wherein said area is stored directly insaid image by replacing pixels within the image indicative of red-eyephenomenon to create said modified image.
 10. The apparatus of claim 8,wherein said area is stored as an overlay of said image thus preservingthe original said image.
 11. The apparatus of claim 10, wherein saidarea is processed by an external device.
 12. The apparatus of claim 11,wherein said external device comprises a personal computer.
 13. Theapparatus of claim 11, wherein said external device comprises a printer.14. The apparatus of claim 7, further comprising an image output fordownloading an integral image display for printing said image modifiedby said red-eye filter.
 15. The apparatus of claim 7, further comprisingthe red eye filter for generating an overlay for said area indicative ofthe red-eye phenomenon of the captured image for at least one of imagedisplay and image output.
 16. The apparatus of claim 7, wherein the areathat is indicative of the red-eye phenomenon comprises a color and shapeindicative of the red-eye phenomenon and the image is modified to changethe color to a black color and further wherein: the integral flashselectively provides illumination during image capturing; and saidred-eye filter is enabled to modify the image in response to said sourceof light providing illumination during image capturing.
 17. Theapparatus of claim 16, further comprising an exposure control means fordetermining if the image was captured in a condition conducive to thered-eye phenomenon and for generating a red-eye signal in responsethereto, wherein said red-eye filter is further enabled in response tothe red-eye signal.
 18. The apparatus of claim 7, wherein said red-eyefilter further includes a false-detection avoidance apparatus whichenables modification of the area that is indicative of the red-eyephenomenon in response to an absence of color indicative of the red-eyephenomenon within a vicinity of and exclusive to the area.
 19. Theapparatus of claim 7, wherein said red-eye filter further includes afalse-detection avoidance apparatus which enables modification of thearea in response to one or more of a substantially white colored region,an iris ring and an eyebrow line within a vicinity of the area.
 20. Theapparatus of claim 7, wherein said red-eye filter detects said areawithin the image indicative of a red-eye phenomenon based on one or moreof a substantially white colored region, an iris ring and an eyebrowline within a vicinity of the area.
 21. The apparatus of claim 7,wherein said red-eye filter comprises: a pixel locator for locating thepixels having a color indicative of the red-eye phenomenon; a shapeanalyzer for determining if a grouping of at least a portion of thepixels located by said pixel locator comprise a shape indicative of thered-eye phenomenon; and a pixel modifier for modifying the color of thepixels within the grouping.
 22. The apparatus of claim 21, furthercomprising a false-detection analyzer for further processing the imagein a vicinity of the grouping for details indicative of an eye, and forenabling said pixel modifier in response thereto.
 23. The apparatus ofclaim 21, further comprising an exposure analyzer for determining if theimage was recorded in a condition indicative of the red-eye phenomenon.24. The apparatus of claim 21, wherein the red-eye filter furthercomprises an exposure analyzer for determining if the image was recordedin a condition indicative of the red-eye phenomenon.
 25. The apparatusof claim 21, further comprising an exposure analyzer for determining ifthe image was recorded in a condition indicative of the red-eyephenomenon including determining whether said flash was used duringimage recording.
 26. The apparatus of claim 21, further comprising anexposure analyzer for determining if the image was recorded in acondition indicative of the red-eye phenomenon including determiningwhether low ambient lighting conditions existed during image recording.27. The apparatus of claim 21, further comprising an exposure analyzerfor determining if the image was recorded in a condition indicative ofthe red-eye phenomenon, and a distance analyzer for determining if thesubject was at a relatively close distance to the apparatus during imagerecording.
 28. The apparatus of claim 7, said meta-data informationbeing analyzed in said red-eye phenomenon determination prior to a pixelanalysis.
 29. The apparatus of claim 7, said meta-data informationcomprising digital apparatus-specific information.
 30. The apparatus ofclaim 29, said meta-data information comprising information regardingconditions under which the image was acquired.